Oracle Complexity Separation in Convex Optimization

Many convex optimization problems have structured objective functions written as a sum of with different oracle types (e.g., full gradient, coordinate derivative, stochastic gradient) and arithmetic operations complexity these oracles. In the strongly case, also condition numbers that eventually define iteration first-order methods number calls required to achieve given accuracy. Motivated by desire call more expensive oracles fewer times, we consider problem minimizing two propose generic algorithmic framework separate complexities for each function. The latter means function is called times coincide case when second absent. Our general accelerated covers setting (strongly) objectives, both parts are through oracle, well one them derivative or has finite-sum structure available gradient oracle. cases, obtain random descent variance reduced separation.